Equilibrium Properties of Temporally Asymmetric Hebbian Plasticity 
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TAH plasticity is described as a change to the synaptic 
efficacy w between two cells. A pair of spikes in the input 
cell and the output cell, at times U and t a , respectively, 
induces a change in w: 



±Xf±(w)K(\t -U\). 
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A theory of temporally asymmetric Hebb (TAH) rules which depress or potentiate synapses 
depending upon whether the postsynaptic cell fires before or after the presynaptic one is presented. 
Using the Fokker-Planck formalism, we show that the equilibrium synaptic distribution induced 
by such rules is highly sensitive to the manner in which bounds on the allowed range of synaptic 
values are imposed. In a biologically plausible multiplicative model, we find that the synapses in 
asynchronous networks reach a distribution that is invariant to the firing rates of either the pre- 
or post-synaptic cells. When these cells are temporally correlated, the synaptic strength varies 
smoothly with the degree and phase of synchrony between the cells. 

PACS numbers: 87.18. Sn, 87.10. +e, 05.10Gg 

Recent experimental evidence indicates that synaptic 
modification in cortical neurons depends on the precise 
temporal relation between pre- and postsynaptic firing 
[0,1). Presynaptic spikes that precede postsynaptic fir- 
ing lead to synaptic potentiation, while those that follow 
postsynaptic firing elicit synaptic depression. The tem- 
poral window for inducing these changes is on the order 
of 10 msec. Several recent theoretical studies addressed 
the potential implications of this temporally asymmet- 
ric Hebbian (TAH) synaptic plasticity on learning || ||. 
The present study is motivated by recent work by Ab- 
bott and coworkers who applied TAH learning in a large 
population of excitatory presynaptic cells asynchronously 
driving a single postsynaptic cell ||. Their simulations 
showed that the distribution of synapses converged to a 
bimodal distribution. The synapses were either almost 
zero or had values close to their upper limit. More- 
over, when the firing rate of the presynaptic cells was in- 
creased, the number of strong synapses decreased so that 
there was very little change in the output rate. Thus, 
the TAH rule seems to provide a mechanism for keeping 
the mean output rate invariant. Since Hebb rules are 
presumed to underlie many developmental and learning 
processes in neuronal systems, it is important to under- 
stand the equilibrium properties of networks with TAH 
plasticity and how they depend upon the particular im- 
plementation of these rules. 

In this Letter, we study the TAH rule using Fokker- 
Planck theory ^|J|] . Surprisingly, we find that the behav- 
ior of the system depends crucially on how the boundaries 
on the allowed range of synaptic efficacies are incorpo- 
rated. In particular, the salient features found in [0 are 
unique to an additive learning rule in which the mag- 
nitude of the update does not explicitly depend on the 
current value of the synapse. A very different behavior is 
found with a multiplicative rule where the magnitude of 
the update decreases as either the upper or lower bounds 
are approached. 
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The weight w is increased by A + w when t D > ti and de- 
creased by A~w when U > t Q . The temporal dependence 
of the update is defined by the filter K which for sim- 
plicity is taken to be K(t) = exp(— t/r). The coefficient 
A sets the scale of the synaptic change at each update. 
The factors f±(w) determine the relative magnitude of 
the changes in the positive and negative directions. 

We consider two particular examples of these update 
rules. The first is an additive update rule where the mag- 
nitude of the changes is independent of w, so that: 



/+ = !; /- = oi. 



(2) 



The parameter a > denotes a possible asymmetry be- 
tween increasing and decreasing the synaptic efficacy. 
If the update results in a synaptic weight outside the 
bounds < w < 1, the weight is clipped to the boundary 
values. For the second example, which we will call the 
multiplicative rule: 



f+(w) = 1 — w; f-(w) = aw. 



(3) 



This results in a synaptic increase (decrease) whose mag- 
nitude scales linearly with the distance to the upper 
(lower) boundary, similar to the model in 

To evaluate the equilibrium properties of these rules, 
the firing activity in the two cells needs to be specified. 
We consider the case when the input and output activity 
are stationary stochastic processes. The firing of the in- 
put cell is characterized by an instantaneous rate function 
— Tit- ~ ^t)j where U are the spike times of the 
input cell with mean rate (i/j) = r,„. Similarly, the activ- 
ity of the output cell is given by v a {t) = J2 t 8{t—t ) with 
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mean rate r ou t . The correlation between these two spike 
trains is described by the normalized time delayed cross- 
correlation function C(t) = {vi{t')v {t' + t)) /r in r out — 1. 
Note that for uncorrected spike trains, C(t) = 0. 

In the limit of small step sizes (A << 1), Eq. (0) can 
be averaged in order to describe the behavior of w on 
times of order 1/A as a continuous random walk, similar 
to the approach in || (see also 0). This random walk 
has a mean drift: 



(a) 



(b) 



,dw. 



[(/+-/-)t + /+T + -/_T_] (4) 



where the weighted correlation times, T±, are 



T+ = 



dtK(t)C(±t). 



(5) 



The first term in Eq. (0) is the contribution from uncor- 
related firing activity in the cells and is proportional to 
t = J °° dtK(t). The other terms represent the contri- 
bution from the synchrony between the two spike trains 
and are proportional to T± . The expression for the diffu- 
sion constant D(w) of this random walk is more complex 
and will be presented elsewhere. Here we note that D 
is small since it is proportional to the A. According to 
Fokker-Planck theory, the equilibrium density P(w) can 
be described as a Gibbs distribution with a plasticity po- 
tential U(w) where in the limit of small A: 



U(w) = -Alog[P(i 



;)] « -2 / 
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dw'v{w')/D{w') (6) 



Thus, P(w) will be concentrated near the global min- 
ima of U(w). Depending upon the implementation of 
the model, the minimum can be located at an interior 
point where the drift v(w) vanishes, or at the boundaries 
w = and w — 1. To evaluate whether the distribution 
of w contains a peak at < w < 1 or at the boundaries, 
the specific form of the correlation function C(±t) needs 
to be considered. 

We first consider the simple example where the spike 
train {ti} is a homogeneous Poisson process and the out- 
put spike train is a shifted version of the input train, 
i.e., t a = ti + At where At is the temporal shift between 
the two spike trains. In this case, r — r\ n = r out , and 
C{t) = r-^it-At), and T± = r' 1 exp(-|At|/r)0(±A£), 
where 6(x) = 1 if x > and otherwise. For the additive 
model in Eq. (0), this leads to a net drift: 



(1 — a)rr 2 
(1 — a)rr 2 



re~ At / T , At > 
are At / T , At < 0. 



(7) 



Here v (as well as D) is independent of w and the poten- 
tial U(w) is U(w) » —2vw/D. In the limit of small A, 
the equilibrium distribution will be a 5-function centered 
at when v < and at 1 if v > 0. 
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FIG. 1. Simulation results showing the mean synaptic effi- 
cacy (w) of a single synapse undergoing (a) additive and (b) 
multiplicative TAH plasticity. At is the time delay imposed 
on the spike times of the post synaptic cell relative to those of 
the pre synaptic ones. All simulations are with r = 10 msec 
and A = 0.005. 

These results are confirmed by the simulations shown 
in Figure 1(a) and 2(a), where we have taken a = 1.05 
and 0.95. For a = 1.05, the magnitude of the nega- 
tive change is slightly larger than the positive one. The 
mean synapse is zero except when < At < Ato with 
Ato w 50 msec. In this range, the positive correlation 
between the input and output cells overcome the nega- 
tive bias in the update rule to generate a positive drift 
so that w « 1. The transition at Ato 1S precisely the 
point where T + = (a — l)r, see Eq. (Q). This behavior 
is highly sensitive to whether a is larger or smaller than 
1. For a = 0.95, the mean synapse is at zero only in the 
range —Ato < At < where the negative correlation is 
larger than the positive bias. Otherwise iswl. 

In contrast, for the multiplicative model the drift ve- 
locity is given by: 

[1 - (1 + a)w] tt 2 + r(l - w)e- At / T , At > 
[1 - (1 + a)w]Tr 2 - awre At / T , At < 

(8) 

Here the drift depends on w. It is positive for small w 
and becomes negative for large values of w. In this case, 
U has an approximately parabolic shape with a minimum 
located at w = wo where the drift velocity vanishes: 



1 - a [l + a + (Tr)- 1 e- At / T ] 



[l + a(l + (rr) 



-1 At/r 



)]' 



At > 
At < 0. 



(9) 



This leads to a distribution P(w) with a narrow peak at 
Wo, as shown in Fig. 2(b). For large values of At, the 
input and output cells are essentially uncorrelated, in 
which case w = 1/(1 + a) w 0.5 for awl. For positive 
At < 50 msec, the positive correlation between the two 
cells gives rise to a mean (w) > 0.5 as shown in Fig. 1(b). 
Conversely, for small negative At, the reverse corelation 
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leads to a mean (w) < 0.5. Thus, through this learning 
rule, the synapse smoothly encodes the temporal phase 
relationship between the presynaptic and postsynaptic 
cells. A similar dependence is found when one varies the 
degree of synchrony between the two cells rather than its 
phase. 
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FIG. 2. Histogram showing the distribution of synaptic 
efficacies using the (a) additive and (b) multiplicative TAH 
rule with r = 10 msec, a = 1.05, A = 0.005. The upper 
two histograms in each graph shows the behavior of a single 
synapse as described in Fig. 1. The lowest histograms are the 
distribution of synaptic efficacies in a network of N = 1000 
excitatory cells with Poisson activation times and mean input 
rate r; n = 10 Hz. These cells converge on a single integrate 
and fire cell with parameters r m = 20 msec, t s = 5 msec, 
V s = 5, and g s = 0.01. 

Let us now consider the situation where a large pop- 
ulation of N cells with modifiable excitatory synapses 
Wi drive a single postsynaptic cell. In the numerical 
simulations below, the output neuron obeys dynamics 
commonly known as "Integrate and Fire" , where the po- 
tential of the cell is described by the equation: r m V — 
—V — I s . r m is the passive time constant of the cell, and 
when the potential V reaches the threshold V = 1 it is 
reset to zero. I s (t) is the synaptic current generated by 
the N excitatory cells. Each spike in the presynaptic cells 
triggers a contribution to the output cell's conductance 
that decays with synaptic time constant r s , yielding: 



JY 



I s {t) = E eWT'HV - V.). 



(10) 



V s is the reversal potential for the excitatory synapses. 
The synaptic efficacy Wi(t) describes the increase in the 
output cell's conductance in units of g s , immediately af- 
ter a spike in the i-th cell. The peak conductances, Wi, 
are in turn modified by the TAH dynamics described 
above. 
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FIG. 3. The effect of increasing the input rate, ri„ on 
the equiiibrium state of a network of N = 1000 excitatory 
synapses driving an integrate and fire ceif, undergoing (a) ad- 
ditive and (b) muitipiicative TAH dynamics. Shown are the 
output firing rate r out , the mean synaptic efficacy (w), and 
the correlation strength \- The latter was determined by fix- 
ing one of the synapses to 0.5 and numerically integrating 
Eq. (||) and using Eq. (0). 

Here we present a general theoretical analysis of the 
system which is independent of the details of the output 
cell dynamics. In the limit of small A Eqs. (§|)-(||) holds 
for each of the i-th synapses, with its correlation times Tj_ 
and T ± l defined using the correlation Ci{t) between the 
i-th cell and the output. As before, we will assume that 
the inputs are described by independent homogeneous 
Poisson processes, all with mean rate r,„. However, the 
statistics of firing in the output cell as described by its 
mean rate r out and {Ci(t)}fL 1 , is determined by its re- 
sponse to the incoming spikes rather than determined 
externally as in the previous example. We first describe 
the behavior of the additive model. Due to causality and 
the Poisson nature of the input spikes, T l _ — 0. For Ti, 
we make the following plausible assumptions (i) since the 
output cell is driven by a large number of asynchronous 
inputs, Ti is positive but small; and (ii) its value in- 
creases roughly linearly with the synaptic efficacy of the 
presynaptic i-th cell, namely, 



TI 



fill 



where (iii) x = x( r out) is expected to be a monotonically 
decreasing function of the firing rate of the output cell. 
X is larger at smaller r out when the output cell spends 
more time near threshold and thus is more sensitive to 
the timing of the incoming spikes. Eq. ([ll]) implies that 
the drift for each synapse, Vi oc 1 — a + xwi. Thus, if a — 1 
is positive and of order 1 , the system will converge to a 
state where all the Wi are zero, or when a < 1, io» ~ 1. 
An interesting situation occurs when < a - 1 < 1 so 
that the weak negative bias can balance the weak posi- 
tive correlations. In this case, the diffusion constant D 
is approximately constant, and the potential is given by: 



U(w) 



1 + a 2 



1 2 

(a - l)w - -xw 



(12) 
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which has local minima at the boundaries, w = 0,1. 
This equation has to be solved self-consistently since 
Tout which determines \ is itself dependent on U . Over 
a wide range of input rates the self-consistent solution 
is an unsaturated state, in which P(w) has significant 
weight both near and near 1, which in turn implies 
that U(0) = 17(1) up to order A. Hence by Eq. @, this 

such that 



state is characterized by an output rate r* ut 
X (C t ) = 2(a-l). 



(13) 



More precisely, as rj„ increases, r out increases slightly by 
an amount of order A inducing a decrease in \ of that 
order. This leads to a small relative increase in £7(1), 
which in turn reduces P(l) by an amount which roughly 
compensates for the increase in rj n , maintaining Eq. (13). 
This behavior is confirmed by simulations whose results 
are shown in Fig. 2(a) and 3(a). As r.;„ increases from 
10 to 40 Hz the output rate remains approximately con- 
stant at » 22Hz, and \ « 0.1 in agreement with eq. 
( |l3| ) . The mean efficacy (w) decreases to compensate for 
the increase in rj n , as found in ||. 

The multiplicative model results in a very different 
state. In fact, since the correlations are weak for large 
N, the equilibrium behavior is similar to the previous ex- 
ample with only a single input and output cell with large 
At. In particular, like Eq. (||), the potential U(w) has a 
single minimum at the point of zero drift: 

w Q a -J— (14) 
1 + a 

Thus, the distribution is highly concentrated near Wq, as 
seen in Fig. 2(b), and is largely independent of the mean 
rates of both the input and output cells. As r.;„ increases, 
the output rate also increases and is similar in behavior 
to a cell with fixed synapses, Wi wtoo. Note that as r ou t 
increases, x decreases but this results in only a small de- 
crease in the mean synaptic efficacy. Finally, we note 
that in contrast to the additive model where the bound- 
aries are local minima of U and the point of zero drift is 
a maximum of U, in the multiplicative model U(w) has 
a single minimum at the point of zero drift. Hence, the 
equilibration time of the additive model will in general be 
much slower than that of the multiplicative model since 
synapses have to overcome the potential barrier of the 
synaptic potential. Indeed, this difference in equilibra- 
tion times is seen in our numerical simulations. 

We have shown here that the multiplicative TAH 
rule leads to a very different equilibrium distribution of 
synapses compared with an additive rule. Most impor- 
tantly, the multipicative model is not sensitive to moder- 
ate changes in the parameters of the plasticity rule and 
does not suffer from slow convergence. Furthermore, ex- 
perimental results reveal a dependence of the magnitudes 



of the synaptic changes on the amplitude of the initial 
synaptic efficacy, which supports a multiplicative TAH 
rule 0]. The observed mean fractional negative change 
remained constant over a wide range of synaptic effi- 
cacies, and is consistent with the assumption that the 
negative change is proportional to w as described by a 
multiplicative f-(w) in Eq. (|^). The fractional positive 
changes monotonically decreased with synaptic efficacy 
and vanish smoothly at some maximum value, again in 
qualitaive agreement with the multiplicative model. Al- 
though the observed shape of the w dependence deviates 
from the simple linear form of f+(w) assumed here, the 
qualitative properties of the rule are unaffected by this 
difference. In conclusion, networks with the multiplica- 
tive TAH rule in the asynchronous state should display 
an equilibrium synaptic distribution that is largely in- 
sensitive to the firing rates of the pre- and post-synaptic 
cells. On the other hand, a coherent temporal modula- 
tion of the firing of the inputs to a target cell leads to a 
synaptic distribution which encodes the degree and phase 
of the synchrony between the cells in a smooth manner. 
The functional implications of this behavior in the de- 
velopment and learning properties of neuronal systems 
remains to be explored. 
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